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(54) Motor vehicle occupant detection system employing ellipse shape models and bayesian 
classification 



(57) A object (20) on a seat (1 6) of a motor vehicle 
(10) is classified by creating a video image of the area 
and forming a silhouette of the object. The silhouette is 
divided into two segments and a separate ellipse (64,66) 
is positioned to approximate the shape of each seg- 
ment. The parameters that define the location and size 
of the two ellipses (64,66) form a feature vector for the 
object (20). A Bayesian classification function utilizes 



the feature vector to determine the probability that the 
object fits which each of a plurality of classes. A class 
for the object (20) is detennined based on the probabil- 
ities. This method can be used to control operation of 
an air bag (24) in the motor vehicle (1 0) in response to 
the class of the object on the seat. 
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inlur, , .mall child or Inlan, «„g ,„ L IroJ^^H^^ZT 
Summary Of The Invention 

[0006] The present invention provides a mechanism for detectinq the presence or «h<=«n^o «f on 



Brief Description Of The Drawings 
[0012] 



Figure 1 is a view from above of the forward interior of a motor vehicle- 

Rgure 5 illustrates parameters of a pair of ellipses which have been defined in the image by the processing step 
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depicted in Figure 4. 
Detailed Description Of The Invention 

5 [0013] With reference to Figure 1, the forward portion of the passenger compartment of an automobile 10 has a 
dashboard 12 from which a steering wheel 14 projects. A front seat 16 is spaced from the dashboard 12 and includes 
standard head rests 1 8 for both the passenger 20 and driver 21 . It should be noted that the passenger 20 may be an 
adult, a child or an infant sitting in either a forward or rearward facing infant seat placed on the vehicle seat 16. The 
steering wheel 14 has a compartment which houses a first air bag 22 and a second air bag 24 is concealed In a 

10 compartment of the dashboard In front of the passenger 20. 

[0014] A video camera 26 is mounted at the ceiling of the passenger compartment near the horizontal center of the 
windshield. The video camera 26 is aimed downward toward the seat area nomnally occupied by the passenger 20, 
as indicated by dashed lines, and produces a generally side view image of the passenger Any of several commonly 
available video cameras may be used, such as a charge coupled device (CCD) imager or an active pixel sensor (APS). 

15 The video camera 26 can be responsive to visual light, but preferably is responsive to near infra-red light from sources 
(not shown) that are mounted above the windshield to illuminate the front seat 1 6 of the vehicle. Use of infra-red light 
allows operation of the present system at night without affecting an occupant's ability to see the road. In addition, the 
illumination fills-in image shadows during daytime operation. 

[0015] With reference to Figure 2, the images produced by the video camera 26 are applied to a video input circuit 

20 32 of an image processor 30. Each video image consists of a two-dimensional array of picture elements (pixels). The 
video input circuit 32 controls storage of the pixels in a random access memory (RAM) 34 with both devices being 
connected to a set of signal buses 36. The RAM 34 also stores variable values, intermediate data, and final results 
produced the Image analysis, as will be described. The analysis of the acquired images is controlled by a microproc- 
essor 38 which executes a program that is stored in a read only memory (ROM) 40. 

r5 [001 6] A control interface circuit 42 also is connected to the set of signal buses 36 and interfaces the image processor 
30 to other components in the motor vehicle 1 0. In the exemplary system, the image analysis controls operation of the 
air bags 22 and 24 and the control interface circuit 42 is connected to the air bag controller 44 for the vehicle. The air 
bag controller 44 also receives a signal from a conventional collision sensor 46 when an accident occurs. The air bag 
controller 44 responds to these inputs by activating the driver and passenger air bags 22 and 24. 

30 [0017] With reference to Figure 3, the image analysis commences each time a passenger sits in the right side of the 
front seat 1 6 of the motor vehicle 1 0. For example, microprocessor 38 receives a signal on Input line 48 which indicates 
when all of the doors of the vehicle close after any one of them has been open. This can be accomplished by sensor 
switches (not shown) mounted on the jam of each door. Alternatively, the image analysis process can occur during 
operation of the vehicle at periodic intervals defined by a timed interrupt of the microprocessor 38. Regardless of the 

35 mechanism used to trigger image processing, program execution starts with an initialization step 50 at which variables 
and other parameters used in the processing are set to initial values. The program then advances to step 51 where a 
new image of the passenger area of the vehicle is acquired. Specifically, the microprocessor 38 issues a command to 
the video input circuit 32 that causes the camera 26 to produce an image which then is stored pixel by pixel within the 
RAM 34. At this point, each pixel has a numerical value defining the gray scale brightness level for the corresponding 

•to location within the image. 

[001 8] The Image processing then advances to step 52 where the recently acquired gray scale image is transformed 
to produce a silhouette of any object on the seat. Techniques for this transformation are described by W. K. Pratt In 
Dtgitai Image Processing, 2nd Ed., John Wiley & Sons, London, 1991. One technique, first separates the portion of 
the image that corresponds to the occupant from the background by storing a reference image of the empty seat into 

•*s the memory of the image processor 30. That reference image is subtracted on a pixel by pixel basis from a newly 
acquired image to produce a difference image. Pixels in that difference Image have a relatively large value where the 
corresponding portion of the newly acquired image is very different from the reference image of an empty seat. i.e. the 
portion where an object is present on the seat. Next, the pixels of the difference image are compared to a threshold 
value to fonn a binary image by converting the pixels with values above the threshold value to a value of one and 

50 converting the other pixels to zero. In the resultant binary image, pixels of the object have a value of one and the 
remaining pixels are zero, thereby fomriing a silhouette of any object or person that is present on the passenger side 
of the motor vehicle seat 16. 

[0019] In order to classify that object or person, features that reflect the shape of the silhouette must be extracted 
from the image. It is recognized that a person seated in the passenger area of the vehicle will have a generally L- 
55 shaped silhouette, such as shown by silhouette 60 in Figure 4. This silhouette can be characterized by a pair of ellipses 
64 and 66 that approximate the shape of a segment of the object. The first ellipse 64 substantially encloses to the torso 
and head of the passenger 20, while the second ellipse 66 substantially encloses the lower portion of the passenger's 
body. The use of a single ellipse to enclose an object in an image being processed is well known as described by W. 
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representation of the obSt.TshoJL be u^Tod IhLt the ^^^..^'f ""J^^-^^^'-Pes that provide a more precise 
the silhouette is a best-fi approxImaSon onhnirf. L f ^"P'''^^*''^" °' ^^''^ «'"Pse to the respective segment of 
from the main body of the ^O^tZyVo^i^l^^^^^^ t?""'^"* °' " P"'^'^' "^^^^ M 

respective segment of the silhouette" sZZ^^\^t^^CT"^"^T''^ '''"^ f'"'"*' ^^'^'^ ^'''P^^ «<> ^^^^ 

legs and torso of eveor passenger^eSL rt^e t^^ always correspond to the junction between the 

junction is shifted in the image as he seal 6 1. 0^112^ ! I"!' ''"'^^ '^"^^"sions differ among people and that 
Physical junction due to Sen ^ody ^ es Ind reaJpo^^^^^^ "'""^^ ^''^ °' 

inches in a typical automobile) and corrJsponS to alTo^^ ^J^^^'T' " "^^'"^""^ °^ ^''^ 

the fixed demarcation line 62 does notSSi exairo^the^unJS^ h'Sf ""'J'^ ^''^ ^'^^^ 

not Significantly affect the image processing rUuS^ the junction between the passenger's legs and torso does 

'^^LiTorSir^TJl:::^^^^ '° ^-^^'^^^ "'^ ^^^-^ ^"'^°-«« t^^t is on one Side 

different types Of occupani anS^ri^the^ o^^^^^^ '''Tt " ^'^--t-ting between 

For exampte. this allows the Tmage^tce^^^^^^^ T"'^ '° ^^^^ bags. 

dimensional feature vectorhavinq the Daram^rfof inL,«nf ^ "^'^"^ ^' ^^''^ ^'"P^^ ^^""^^ "y a five 

the length (a) of the major axT'^e lenXfw Tf ^e Z^^^ ''T^' <^ y) <=enter in the image, 

horizontal. The image proc^sor 30 coSS th^^^^^ ' ^"'^ ^" ^"^le f between the major axis and Imag^ 
blnedtofo^aten^dlm^fJatrvr^^^^^^^^ 



^ - (^1 , Vi , , , , yg. bg, fg). 

[0023] 



Ob ect class is ignored. The object classification nmroce 1,T. ^ » moment, the empty seat and irrelevant 

preserve, a p*', »/ .sc^i'sr .rcSsr f.«T « PpUCsiTTri: ^ 

scribes how well the ellipse parameters derived fmm tho irrTr,L ■' / ' ^ likelihood function de- 

EVh^rf~^^ 

wh.h hasthe maximum decision ^c^ valu" ! i.eTe reaTe^trb^b^r"^ ^ " ^'^^ 

[0026] The optima fomi of the decision function that minimizes the probinity of classification error is given by 



= log P;t(x)+log P[k]. k = 1 ,2.3,4 



(2) 



P,(X) = (271)-''°^ |C J-''^ ^-1/2(x.m,)^ c,^ (x-m,) 
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where m^f and C^^are the mean vector and covariance matrix, respectively, for class k. 

[0027] The mean vector and covariance matrix for every possible class are estimated from training Images. In the 
case of the air bag control system, a plurality of different objects in each class are placed on the seat and an Image of 

each one is produced. Suppose that , Xg, X3 x^ are the two-ellipse feature vectors extracted from n training images 

of the adult class (k=3), then estimates of the mean vector m^ and covariance matrix are given by: 



Thus a set of templates is developed for a typical silhouette in each class. 

[0028] Thus the application of the Bayesian classifier function to the feature vector x produces a probability that the 
present silhouette fits within each of the object classes. A preliminary determination as to the type of object on the seat 
{adult, a child, a forward facing Infant seat,) or a reanward facing Infant seat) then is reached based on the class having 
the greatest probability. 

[0029] However, there remains the situation where the seat is empty or contain an irrelevant object, such as a grocery 
bag, which does not fall within any of the expected classes. If the Bayesian classifier assigns the ten dimension feature 
vector to the first class kQ (for example the FFIS object class), then d^x) achieves a maximum value for )t equal to k^. 
However, if the probability df^Jix) is smaller than a predefined threshold value, a determination is reached that there is 
insufficient evidence that the ten dimension feature vector belongs to any one of the four classes. In this case, the 
reasonable determination is to assign the feature vector to the empty and the irrelevant object class. 
[0030] Once the dass of the object has been detemained at step 58, that Infonnation can be used to formulate a 
command that is sent at step 59 to the control interface 42 to provide infonnation to the air bag controller 44. For 
example, if the air bag controller Is infonned that a forward facing infant seat (FFIS) or a rear facing infant seat (RFIS) 
is present on the seat, the air bag controller 44 inhibits inflating the passenger side air bag 24 in the event of an accident. 
Similarly, If a detemiination is made that a child may be present in the passenger seating area, the air bag controller 
44 inflates the air bag during an accident at a much slower rate than for an adult. 



1. A method for classifying a object (20) that is present within a motor vehicle (10), that method comprising: 

acquiring an image (60) of the object; 

segmenting the object Into a first segment and a second segment; 

approximating the first segment with a first geometric shape (64) which is selected from a group consisting of 
a circle, an ellipse, and a regular polygon and which is defined by a first set of parameters; 
approximating the second segment with a second geometric shape (66) which Is selected from a group con- 
sisting of a circle, an ellipse, and a regular polygon and which is defined by a second set of parameters; 
fonning a feature vector from the first and second sets of parameters; 

comparing the feature vector to plurality of template vectors, each representing different class of objects, to 
determine probabilities that the object falls within each class; and 
classifying the object in response to the probabilities. 

2. The method as recited in claim 1 wherein comparing the feature vector to plurality of template vectors utilizes a 
Bayesian classification function. 

3. The method recited In claim 1 further comprising classifying the object (20) in a given object class when none of 
the probabilities exceeds a predetenmined value. 
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acquiring an image of an interior of the motor velnicle- 

extracting a portion of the image which corresponds to an object (20) on the seat (16V 
dividing the object into a firet segment and a second segment- ^' 
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stJ^paiTe^^^^^ 

fonning a feature vector from the first and second sets of parameters 
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side of a demarcation line, and defines the second segment as a portion of the object on another side of a demar- 
cation line. 

19. The method as recited in claim 1 2 further comprises defining each of the plurality of template vectors by producing 
a feature vector from at least one exemplary object in a respective class of objects. 

20. The method recited in claim 1 further comprising classifying the object Is a given object class when none of the 
probabilities exceeds a predetemnlned value. 



10 



15 



20 



25 



30 



35 



40 



45 



SO 



55 



7 



BNSDOCID: <BP. 



.1251465A2J_> 



EP 1 251 465 A2 




FIG. 2 
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FIG.3 (START) 
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